Original  article 


JDMS 


Statistical  Modeling  of  Combat  Mortality 
Events  by  Using  Subject  Matter  Expert 
Opinions  and  Operation  Iraqi  Freedom 
Empirical  Results  from  the  Navy-Marine 
Corps  Combat  Trauma  Registry 


Journal  of  Defense  Modeling  and 
Simulation:  Applications, 
Methodology,  Technology 
XX(X)  I- 1  I 

©  20 10  The  Society  for  Modeling 
and  Simulation  International 
DOI:  1 0. 1  1 77/ 1 5485 1 29 1 0364755 
http://dms.sagepub.com 

<§)SAGE 


Ray  Mitchell1,  Joseph  Parker1,  Michael  Galarneau2  and 
Paula  Konoske2 


Abstract 

In  12  Navy-Marine  Corps  medical  logistics  studies  and  analyses  conducted  by  the  Naval  Health  Research  Center  (NHRC) 
and  Teledyne  Brown  Engineering  (TBE)  over  the  past  5  years,  estimates  of  battlefield  mortality  have  been  a  major  metric 
of  interest  to  medical  planners.  An  ongoing  concern  is  how  mortality  is  related  to  delays  in  treatment  for  medical  logistics 
reasons.  In  this  paper,  we  describe  how  NHRC  and  TBE  have  been  developing  a  statistically  based  model  for  mortality 
since  2003,  first  starting  with  panel  results  from  a  group  of  military  medical  doctors  and  continuing  here  with  an  analysis 
of  empirical  injury  data  from  Operation  Iraqi  Freedom  (OIF). The  panel  results  and  statistical  analysis  of  life-threatening 
injury  data  in  OIF  from  early  2004  to  mid-2006  indicate  that  the  Weibull  distribution  describes  the  timing  of  high  risk  of 
mortality  events  in  a  reasonable  manner  within  surgical  medical  treatment  facilities. The  quest  for  a  best-fitting  probability 
distribution  with  parameters  dependent  on  the  casualty  flow  chain  of  treatment  and  evacuation  in  the  theater  is  ongoing. 
In  reality,  combining  analytical  and  subject  matter  expert  (SME)  results  to  model  mortality  is  necessary  given  the  breadth 
of  theater  medical  delivery  systems  and  general  paucity  of  adequate  empirical  data  in  many  of  the  segments  of  patient  flow. 
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I.  Introduction 

The  Naval  Health  Research  Center’s  (NHRC)  Tactical 
Medical  Logistics  (TML+)  planning  tool  aids  the  medical 
planner  by  simulating,  in  a  systems  sense,  a  broad  range  of 
stochastic  events  associated  with  a  battlefield  casualty’s  dis¬ 
position  from  the  point-of-injury  (POI)  to  definitive  care 
(http://www.tmlsim.com).  As  designed  and  developed  during 
FY01  to  FY07,  TML+  is  a  software  program  for  medical 
planners  that  estimates  medical  resource  requirements  using 
data  from  the  Estimating  Supplies  Program  (ESP).1  It  per¬ 
mits  a  broad  range  of  operational  risk  assessments,  medical 
systems  analysis,  and  operations  research  studies  to  be  con¬ 
ducted  for  a  variety  of  scenarios.  The  tool  assumes  a  systems 
view  of  the  tactical  Medical  Treatment  Facility  (MTF) 
network  within  the  theater,  where  the  MTFs  are  integrated 
with  transportation  assets  and  compete  for  medical/logistics 
resources  (staff,  equipment,  consumables,  transporters)  as 
casualties  flow  through  the  system.  Other  recent  modeling 


and  simulation  (M&S)  work  of  a  similar  combat  medical 
logistics  systems  nature  is  described  by  von  Tersch  et  al.2 

The  TML+  output  metric  that  has  been  of  paramount 
interest  in  12  theater  medical  planning  studies  over  the  past 
5  years  is  the  died-of-wounds  (DOW)  due  to  a  delay  in 
treatment  mortality  estimate.  ’  Typical  questions  of  interest 
have  included  the  following: 
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1.  How  many  lives  can  be  saved  if  a  forward  surgery 
system  is  present  near  the  combat  area? 

2.  What  is  the  best  way  to  integrate,  in  time  and  space, 
transportation  and  MTF  assets  to  reduce  casualty 
losses  in  a  deployed  network  of  care? 

3.  How  do  spikes  in  casualty  occurrences  impact  oper¬ 
ating  room  throughput  and  mortality  for  various 
staff  and  equipment  configurations? 

The  primary  objective  of  this  research  report  is  to  docu¬ 
ment  NHRC’s  ongoing  effort  to  develop  a  statistically  based 
model  for  TML+  and  other  medical  systems  analysis  models 
that  projects  the  effects  of  treatment  delays  and  the  extent  of 
resuscitation  available  (surgical-level  resuscitation  versus 
non-surgical-level  resuscitation)  on  battlefield  mortality. 
This  empirically  based  capability,  perhaps  merged  with 
available  subject  matter  expert  (SME)  results,  would  allow 
medical  planners  to  more  confidently  estimate  the  mortality 
risk  associated  with  candidate  operational  courses  of  action 
(COAs)  for  either  deliberate  planning  in  the  future  or  for 
crisis  action  planning  for  real-time  deployments. 

2.  Background 

For  use  in  NHRC’s  TML+  discrete  event  simulation,  it  is 
assumed  that  a  seriously  wounded  casualty’s  time  to  death 
is  a  random  variable  that  has  a  probability  density  function 
(pdf)  with  parameters  dependent  on  the  casualty’s  injury 
extent,  current  MTF  capability,  and  past  treatment  history/ 
timing.  In  the  model,  simulated  draws  are  made  from  the 
pdf  throughout  a  scenario  to  interject  mortality  events.  The 
Weibull  pdf,  to  be  described  later,  is  currently  used  in 
TML+  based  on  an  analysis  of  expert  opinion  data  obtained 
from  a  panel  of  nine  military  medical  doctors  convened 
at  NHRC  in  November  2003. 4-7  The  work  by  von  Tersch 
et  al.2  describes  a  mortality  model  that  is  graphical  in  nature, 
and,  indeed,  is  similar  to  a  subjective  algorithm  used  in 
TML+  in  an  earlier  version  in  the  2000  timeframe.8  As 
empirical  injury  records  are  now  available  for  Operation 
Iraqi  Freedom  (OIF)  combat,  as  will  be  introduced  in  the 
next  section,  we  seek  a  more  objective  data-driven  stochas¬ 
tic  model  for  use  in  TML+  and  one  perhaps  of  interest  to 
other  medical  M&S  efforts. 

For  the  modeling  approach  used  in  TML+,  an  obvious 
research  goal  is  to  confirm/update  the  expert  panel  results 
with  a  statistical  analysis  of  empirical  mortality  results. 
A  prerequisite  of  this  analysis  is  a  large  number  of  recorded 
life-threatening  (LT)  incidents  from  an  operational  scenario 
with  sufficient  timing  information  on  treatment  entry  and 
disposition.  Real-world  casualty  resuscitative  data  from  the 
Navy-Marine  Corps  CombatTrauma  Registry  Expeditionary 
Medical  Encounter  Databases  (NMC  CTR  EMED)  on  OIF 
injuries  are  used  to  examine  the  efficacy  of  confirming 
and/or  supplementing  the  panel  results  with  a  time-based 


mortality  analysis.9  The  analysis  file  available  covers  the 
period  from  early  2004  to  mid-2006  and  contains  1079 
injury  records  deemed  to  be  of  a  LT  nature.10  While  certain 
results  applicable  to  TML+  were  obtained  and  will  be  illus¬ 
trated,  several  inherent  limitations  in  the  NMC  CTR  data 
structure  preclude  a  full  investigation  of  battlefield  mortal¬ 
ity  as  a  function  of  detailed  casualty  flow  paths  and  delays 
in  treatment  as  assumed  in  TML+. 1 1  These  limitations  will 
be  addressed  as  topics  for  future  research. 

This  paper  is  organized  as  follows:  a  brief  description  of 
the  SME  panel  results  is  given  in  Section  3;  statistical  anal¬ 
ysis  results  of  NMC  CTR  OIF  mortality  events  using 
techniques  from  the  biostatistics  and  applied  life  data  anal¬ 
ysis  literature  for  certain  resuscitative  capable  MTFs  are 
given  in  Section  4;  a  graphical  comparison  with  SME  panel 
results  is  given  in  Section  5;  and,  finally,  suggested  future 
activities  to  extend  this  research  to  more  completely  model 
mortality  in  terms  of  treatment  delays  and  additional  casu¬ 
alty  flow  variations  are  discussed  in  Section  6. 

3.  Initial  Modeling  Results  Inferred  from 
Subject  Matter  Expert  Opinions 

Figure  1  shows  subjective  observations  and  estimated  results 
from  a  2003  medical  doctor  SME  panel5,7  for  a  patient  with 
a  high  risk  of  mortality  injury.  In  the  figure,  the  casualty  is 
presumed  to  receive  a  series  of  medical  interventions  for  a 
LT  injury: 

1 .  by  self  or  a  buddy  at  the  point  of  injury  (labeled  ‘no 
treatment’); 

2.  by  a  field-level  corpsman  (first  responder)  after  a 
10-minute  delay; 

3.  by  the  Battalion  Aid  Station  (BAS)  MTF  after  a 
30-minute  delay;  and,  finally, 

4.  by  a  Shock  Trauma  Platoon  (STP)/Forward  Resusci¬ 
tative  Surgical  System  (FRSS)  MTF  after  30  min¬ 
utes  of  delay. 

The  residts  illustrate  how  medical  treatment  improves 
survival  as  the  casualty  moves  through  the  MTF  system. 
Curves  of  this  nature  are  used  in  TML+  to  simulate  a  time  of 
death  as  a  seriously  wounded  patient  flows  through  the  bat¬ 
tlefield  medical  chain  of  treatment  and  evacuation.1"  The 
estimated  results  in  the  figure  presume  a  Weibull  pdf,  where 
the  parameters  are  estimated  by  the  ad  hoc  analytical  method 
of  matching  a  few  percentile  points  (Elandt- Johnson  and 
Johnson,13  p.  182).  The  more  standard  analytical  method  of 
maximum  likelihood,  including  all  of  the  observations,  will 
be  used  for  the  empirical  results  to  be  described  later. 

By  convention,  we  define  LT  injuries  as  those  where  a 
casualty  is  expected  to  die  within  the  first  hour  after  injury 
(the  ‘golden  hour’)  if  no  treatment  beyond  first-aid  is 
received  by  either  the  individual  casualty  or  from  a  ‘buddy’. 
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for  the  random  occurrence  of  injury  types  in  treatment  and 
evacuation  scenarios  of  interest.  These  data  sets  provide  the 
empirical  means  to  confirm  or  augment  medical  logistics 
modeling  assumptions  to  better  quantify  the  optimal  mix  of 
health-care  facilities  and  providers,  medical  equipment  and 
supplies,  and  transportation  assets  affecting  battlefield  medi¬ 
cal  delivery.  In  this  section,  we  explore  the  utility  of  using 
NMC  CTR  records  in  a  statistical  analysis  to  provide  a  prob¬ 
ability  distribution  for  the  simulated  timing  of  deaths  within 
a  MTF,  a  distribution  perhaps  dependent  on  prior  treatment 
paths  as  currently  modeled  in  TML+. 

Table  1  shows  a  screen  shot  of  a  very  small  subset  of  the 
CTR  data  file.  The  timing  of  injury  events  and  MTF  entry/ 
exits  along  with  the  disposition  data  is  of  primary  interest  in 
the  file  but  is  suppressed  in  the  table  to  avoid  privacy  issues. 
The  Injury  Severity  Score  (ISS)16  is  one  of  the  many  ele¬ 
ments  in  this  file  for  an  individual  injury  and  is  the  basis  for 
LT  records  we  will  analyze. 


Figure  I.  Expert  panel  survivability  opinions  and  initial  Weibull 
fit  for  a  seriously  injured  patient 


In  TML+,  LT  injuries  are  identified  through  patient  condi¬ 
tions  (PCs)  that  are  designated  as  having  either  a  high, 
medium,  or  low  risk  of  mortality  within  the  first  hour  after 
wounding,  where  the  probability  of  dying  is  greater  than 
2/3,  between  1/3  and  2/3,  or  less  than  1/3,  respectively.5'14 

The  panel  was  conducted  in  a  Delphi-similar  manner, 
where  a  presumed  number  of  battlefield  casualties  with  var¬ 
ious  PCs  developed  by  the  Defense  Medical  Standardization 
Board  and  used  in  NHRC’s  M&S  efforts  were  assumed  to 
be  initially  injured.15  For  each  PC  and  treatment  assump¬ 
tion,  the  panel  was  asked  to  estimate  the  fraction  of  casualties 
that  would  be  expected  to  survive  after  specified  time 
epochs,  such  as  10  minutes,  30  minutes,  1  hour,  3  hours, 
etc.,  in  the  various  interventions  or  MTFs.  An  average  value 
of  their  opinions  and  the  Weibull  estimates  are  shown  in 
Figure  1.  The  rather  impressive  matching  of  these  expert, 
yet  qualitative,  opinion  results  with  the  Weibull  distribution 
is  encouraging  for  the  pursuit  of  a  stochastic  representation 
and  was  judged  adequate  for  an  initial  implementation  in 
TML+.  Similar  curves  for  medium  and  low  risks  of  mortal¬ 
ity  were  obtained. 

4.  Statistical  Analysis  Results  from 
Navy-Marine  Corps  Combat  Trauma 
Registry  Data 

The  NMC  CTR  is  a  data  warehouse  composed  of  data  sets 
describing  events  that  occur  to  individual  casualties,  from 
the  POl  through  the  medical  chain  of  evacuation  and  on  to 
long-term  rehabilitative  outcomes.9  The  CTR  can  assist  med¬ 
ical  planners,  systems  analysts,  and  logisticians  in  planning 


4. 1  Using  the  ISS  as  a  Metric  for  the  Risk  of 
Mortality 

In  using  the  ISS,  it  is  assumed  that  values  less  than  9  corre¬ 
spond  to  injuries  that  are  not  life-threatening  and  values  of  9 
or  larger  correspond  to  life-threatening  injuries.  Furthermore, 
life-threatening  values  between  9  and  14  correspond  to  a 
low  risk  of  mortality,  values  between  15  and  24  correspond 
to  a  medium  risk,  and  values  of  25  and  above  correspond  to 
a  high  risk.  The  OIF  data  file  with  ISS  values  greater  than  or 
equal  to  9  was  slightly  over  1000  records.  Of  these,  about 
80%  corresponded  to  patient  records  at  MTF  level  Ha  (surgi¬ 
cal)  facilities.  The  approximately  20%  of  records  remaining 
were  from  several  MTFs  (BAS  and  STP),  none  of  which 
taken  separately  represented  a  large  enough  sample  judged 
to  be  adequate  for  a  statistical  analysis. 

Too  few  surgical  facility  arrivals  had  detailed  informa¬ 
tion  on  time  of  wounding  or  any  record  of  prior  treatment 
to  permit  a  reasonable  statistical  analysis  of  the  mortality 
effects  due  to  specific  paths  or  delays  in  arrival  (original 
goals).  However,  the  patients  with  complete  timing  infor¬ 
mation  on  arrivals  and  departures  did  allow  an  investigation 
of  the  nature  of  the  random  variable  for  the  time  to  death  in 
a  surgical  facility.  The  effects  of  delays  in  treatment  and  the 
effects  within  a  MTF  are  both  important  in  modeling  casu¬ 
alty  mortality  in  a  simulation  tool.  It  is  expected  that 
additional  records  to  allow  an  empirical  study  of  the  treat¬ 
ment  delay  effects  will  be  available  in  the  future. 

For  the  subset  of  high-risk  patients  with  sufficient  timing 
information  on  arrivals  and  departures,  some  160  records 
with  ISS  values  >25  were  available  across  all  surgical  facili¬ 
ties.  Of  these,  26  (17.1%)  were  labeled  DOWs  fora  surgical 
facility  disposition,  and  the  remaining  134  were  labeled  as 
evacuations  to  the  next  higher  MTF.  Corresponding  groups 
of  medium-  and  low-risk  patients  contained  less  than 
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Table  I.  A  desensitized  portion  of  the  combat  trauma  registry  (CTR) 


Case  ID 

Medical 

Arrival 

Injury 

Triage 

Injury 

Subjective  Objective 

Disposition 

Evac 

Many 

Treatment 

Method 

severity 

Category 

Category 

Assessment  Plan 

Priority 

more  data 

Facility  ID 

Score 

(SOAP)  notes 

entries 

INI 

Battalion  Aid 

Non-Medical 

34 

Immediate 

Blunt 

Patient  brought  to 

Surgical 

Urgent 

Station  2/6 

Ground 

Fox  Co  Firm  Base 

Company 

after  IED  exploded 

4-6  feet  from  him. 

Charlie 

Marines  on  scene 

state  he  was  thrown 
10-15  feet  from 
force  of  blast. ... 

1  1  12 

Shock  Trauma 

Casualty 

57 

Immediate 

Amputation 

Probable  RPG  round 

Died-of- 

Platoon  / 

Evacuation 

through  abdomen, 

Wounds 

Forward 

(CASEVAC) 

chest  and  arm.  Initially 

(DOW) 

Resuscitative 

system 

treated  at  BAS  with 

Surgical 

valve  dressing  over 

System  2 

chest ... 

1  113 

Shock  Trauma 

Medical  Air 

13 

Immediate 

Penetrating 

20  y/o  ad  USMCJED 

Evacuated 

Urgent 

Platoon  / 

rear  passenger 

Forward 

HMWVV.  C/O  B/L  leg 

Resuscitative 

pain,  no  LOC, ... 

Surgical 

System  7 

1  1  14 

Battalion  Aid 

Casualty 

9 

Immediate 

Perforating 

Patient  suffered  GSW 

Evacuated 

Priority 

Station  2/6 

Evacuation 

to  ...  aspect  of  neck. 

(CASEVAC) 

Wound  approx  5cm 

system 

just  inferior  to 
mandibular  angle, ... 

1  1  15 

Surgical 

Casualty 

26 

Immediate 

Hemothorax 

18  y/o  Marine  -  driver 

Died-of- 

Company 

Evacuation 

of  High-backed 

Wounds 

Charlie 

(CASEVAC) 

Humvee  struck  by 

(DOW) 

system 

blast  of  IED.  Occurred 

30min  prior  to  arrival. 
Per  Corpsman  ... 

40  cases  and  were  judged  too  small  for  consideration.  The 
next  section  presents  a  statistical  analysis  of  these  160  data 
results. 

4.2  Statistical  Results  for  the  ISS-derived  Surgical 
Facility  High-Risk  Records 

If  we  can  reasonably  demonstrate  the  characteristics  of 
mortality  within  a  MTF  to  include  the  shape  of  a  distribu¬ 
tion  via  a  pdf,  we  can  simulate  a  discrete  event  in  TML+  to 
represent  DOW  timing.  Candidates  often  used  in  mortality 
studies  include  the  exponential,  gamma,  lognormal, 
Gompertz,  and  Weibull  distributions. 13  As  the  2003  MD 
SME  panel  results  suggested  the  Weibull  distribution,  it  is 
our  first  choice  here.  If  the  Weibull’s  fit  to  the  data  is 
deemed  unsatisfactory,  other  models  will  be  examined.  The 
life  insurance  industry  typically  uses  actuarial  tables  to 
determine  expected  lifetimes  underlying  policy  premiums 
and  do  not  often  depend  on  simulated  individual  death 


times  as  required  in  a  medical  logistics  systems  analysis 
tool  such  as  TML+.13  Hardware  reliability  and  biomedical 
sciences  applications  typically  use  distributions  from  the 
set  described  above.13'1718 

Figure  2  shows  a  time-ordered  plot  of  the  total  DOWs 
and  evacuations  for  the  ISS  >  25  set.  it  also  shows  the 
number  of  surgical  patients  remaining  at  risk  versus  time, 
computed  by  subtracting  the  total  number  of  DOWs  and 
evacuations  from  the  entry  number  (160).  These  are  called 
progressively  censored  samples  in  the  literature  of  biomed¬ 
ical  sciences  and  life  data  analysis  as  patients  enter  and 
leave  at  differing  times,  and  deaths  are  intermixed  with 
withdrawals  where  the  patient  is  alive.13,17'18 

Throughout,  the  random  life  time  T  is  assumed  to  have  a 
pdf  f[t)  and  a  probability  of  surviving  past  time  t  given  by 
the  survival  function  S(t )  =  Pr[  T  >  ?].  To  correspond  to  the 
MD  SME  panel  observations,  we  examine  here  the  useful¬ 
ness  of  the  Weibull  distribution  in  describing  mortality  in 
the  CTR  data  set  available.  The  density  function  with 
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Total  Died-of-Wounds  (DOWs),  Evacuations  & 
Number  at  Risk  vs  Time 


Figure  2.  Time  histories  for  total  died-of-wounds,  evacuations, 
and  patients  at  risk  (n  =  1 60,  surgical  facility  case) 


parameters  {a,  h}  and  corresponding  survival  function  for 
the  Weibull  are18 

fit,  a,  b)  =  a>0,b>0,t>0  (1) 


and 

S(t)=eOb. 


(2) 


The  hazard  function.  fit),  and  cumulative  hazard  function, 
H(t),  are  also  used  in  the  following.  Their  equations  are 


fit) 


f(t)_  b/_t  \b- 1 
S(t)  a' a/ 


(3) 


and 


,41 

where  fis)  is  integrated  from  s  =  0  to  i.  The  exponential  pdf 
is  a  special  case  of  the  Weibull  pdf,  where  the  shape  param¬ 
eter  b  is  1.0;  its  hazard  rate  is  constant.  The  parameter  a  is 
called  the  scale  parameter. 

The  survival  function,  S(t),  and  hazard  function,  fit), 
each  giving  a  different  view  or  interpretation  of  the  mortal¬ 
ity  process,  will  be  estimated  from  data  plots  and  analytical 
means  in  the  next  section.  (Basically,  the  lengths  of  life 
times  via  the  survival  function  and  the  rate  at  which  deaths 
occur  [given  by  the  hazard  function]  are  different  views  of 
the  same  process.) 

It  is  assumed  that  the  patients  entering  the  surgical  MTFs 
are  a  random  sample  from  the  OIF  severely  injured  popula¬ 
tion,  and  the  injuries  incurred  are  a  random  sample  of  the 
battlefield  wounds  likely  from  OIF  theater  operations.  It  is 
further  assumed  that  the  majority  of  patients  arrive  at  the 


surgical  MTFs  directly  from  first  responder  treatment  after 
a  nominal  delay.  Our  basic  approach  will  be  to  characterize 
the  pdf  of  T  by  exploiting  various  features  of  the  Weibull 
distribution. 

4.3  Weibull  Parameters  via  Graphical  and  Maximum 
Likelihood  Methods  of  Estimation 

A  caveat  first:  the  data  presented  here  are  said  to  be  right 
censored  in  that  the  dependent  variable,  death  time,  for  the 
patients  who  were  evacuated  is  known  to  be  greater  than 
the  time  of  withdrawal,  but  the  true  value  is  unknown. 
These  are  often  called  incomplete  samples.  It  is  assumed 
that  if  these  patients  had  remained  in  a  surgical  facility, 
their  subsequent  death  times  would  be  from  the  same  prob¬ 
ability  distribution  as  that  observed  for  the  DOWs,  i.e.  we 
assume  that  the  censoring  process  is  non-informative  of 
future  survivability.  This  is  a  common  assumption  in  the 
literature  for  graphical  and  analytical  techniques  applied  to 
life  data  of  this  nature  (Nelson,18  p.  315). 

Probability  theory  provides  essential  tools  for  survival 
analyses  of  this  type  of  clinical  data,  but  possibilities  of 
departures  from  theoretical  models  are  so  great  and 
varied  that  considerable  detail  is  typically  required  for 
fitting  distributions  to  observations — it  is  for  these  rea¬ 
sons  that  graphical  methods  are  considered  of  first 
importance  to  compliment  analytical  methods  such  as 
maximum  likelihood  (Elandt-Johnson  and  Johnson,13 
p.7).  Nelson18  further  comments  that  each  method  pro¬ 
vides  information  not  provided  by  the  other,  e.g.,  a  plot 
helps  to  assess  the  validity  of  the  assumed  distribution 
and  of  the  data,  while  analytical  methods  provide  confi¬ 
dence  limits  and  objective  estimates.  This  is  the  approach 
that  we  follow. 

Various  non-parametric  techniques  exist  to  estimate  the 
empirical  survival  function  S(t).  One  of  the  oldest  is  a  so- 
called  actuarial  method  dating  back  some  200  years  that 
was  developed  and  used  to  construct  human  life  tables  with 
grouped  data  by  insurance  companies  for  risk  analysis.  A 
more  modern  technique  uses  individual  ordered  observa¬ 
tions  in  continuous  time.  Each  technique  seeks  to  estimate 
the  probability  of  survival  past  some  point  in  time.  Estimates 
of  the  hazard  function  and  the  standard  error  of  S(t)  are  usu¬ 
ally  provided  for  each  technique.  Example  results  of  these 
methods  were  provided  in  various  NHRC/TBE  technical 
interchange  meetings.19  The  so-called  method  of  Kaplan- 
Meier  in  continuous  time  will  be  used  here  for  a 
non-parametric  estimate  of  S(t)  and  related  functions.1718 
The  analytical  method  of  maximum  likelihood,  assuming  a 
Weibull  distribution  of  life  times,  will  also  be  used  to  esti¬ 
mate  the  parameters  of  S(t). 

Referring  again  to  Figure  2,  the  basic  idea  presented  by 
Kaplan  and  Meier  is  to  order  the  intermixed  death  and  evac¬ 
uation  times  and  compute  the  survival  function  after  each 
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Table  2.  Example  data  showing  the  Kaplan-Meier  estimates  of 
the  survival  function,  S(t),  for  the  first  20  minutes  after  surgical 
facility  entry 


Event  Time 
t,  min 

Event 

0-censor 

1  -complete 

Nbr  Remain 

at  t,  r. 

Kaplan-Meier 
Est  of  S(t) 

2 

DOW 

1 

160 

0.994 

6 

DOW 

1 

159 

0.988 

7 

DOW 

1 

158 

0.981 

8 

DOW 

1 

157 

0.975 

10 

DOW 

1 

156 

0.969 

1  1 

DOW 

1 

155 

0.963 

15 

DOW 

1 

154 

0.956 

15 

Evac 

0 

153 

0.956 

15 

Evac 

0 

152 

0.956 

15 

Evac 

0 

151 

0.956 

15 

Evac 

0 

150 

0.956 

15 

Evac 

0 

149 

0.956 

15 

Evac 

0 

148 

0.956 

15 

Evac 

0 

147 

0.956 

15 

Evac 

0 

146 

0.956 

15 

Evac 

0 

145 

0.956 

16 

Evac 

0 

144 

0.956 

17 

Evac 

0 

143 

0.956 

19 

DOW 

1 

142 

0.950 

20 

Evac 

0 

141 

0.950 

death  using  the  recursive  formula  given  below.  Specifically, 
the  Kaplan-Meier  estimate  of  S  after  each  death  event  at 
time  t  is 

i 

S={r  -  1  )/r  *S  ,S  =  1  (5) 

i  v  j  '  i  i  1’  0  K  ’ 

where  r  is  the  reverse  rank  of  the  /th  death.  If  there  are 

i 

assumed  to  be  n  event  times  in  the  sample,  they  are  ordered 
from  smallest  to  largest  and  numbered  backwards  with 
reverse  ranks.  Table  2  illustrates  the  basic  approach  by 
showing  the  DOW  and  evacuation  events  for  the  initial  20 
minutes  within  surgical  MTFs,  where  evacuation  times  are 
labeled  ‘0’  indicating  censoring  or  incomplete  data,  and  ‘1’ 
indicate  death  events,  where  the  timing  is  known.  Estimates 
of  S(t )  are  computed  using  Excel  and  shown  in  the  table. 
Table  3  shows  S(t )  results  after  each  death  event  for  the 
entire  data  set  using  the  SYSTAT  statistical  software  pack¬ 
age.20  It  also  shows  various  other  mortality  metrics  indexed 
by  the  death  times;  the  cumulative  hazard  function  and 
Weibull  entries  will  be  discussed  next. 

The  equation  for  the  cumulative  hazard  function  of  the 
Weibull  distribution  in  (4)  can  be  transformed  into 

log  (0  =  y  log  H(t)  +  log  (a)  (6) 

b 


Table  3.  SYSTAT  computation  for  the  Kaplan-Meier  estimate  of  the  survival  function,  S(t),  and 
other  measures 


Death 
Time 
t,  min 

Number 

At  Risk 

Number 

Dying 

K-M  Est 
of  S(t) 

Std  Error 
of  S(t) 

Hazard 
Rate,  h(t) 

Cumulative 
Hazard,  H(t) 

Weibull  Model 

Survival  Prob 
(via  MLE) 

2 

160 

1 

0.994 

0.006 

0.006 

0.006 

0.995 

6 

159 

1 

0.987 

0.009 

0.006 

0.013 

0.986 

7 

158 

1 

0.981 

0.01  1 

0.006 

0.019 

0.983 

8 

157 

1 

0.975 

0.012 

0.006 

0.025 

0.981 

10 

156 

1 

0.969 

0.014 

0.006 

0.032 

0.977 

1  1 

155 

1 

0.962 

0.015 

0.006 

0.038 

0.975 

15 

154 

1 

0.956 

0.016 

0.006 

0.045 

0.967 

19 

142 

1 

0.95 

0.017 

0.007 

0.052 

0.959 

30 

131 

1 

0.942 

0.019 

0.008 

0.059 

0.938 

32 

124 

1 

0.935 

0.02 

0.008 

0.067 

0.935 

41 

1  1  1 

1 

0.926 

0.022 

0.009 

0.076 

0.919 

43 

1  10 

1 

0.918 

0.023 

0.009 

0.085 

0.915 

44 

106 

1 

0.909 

0.024 

0.009 

0.095 

0.914 

48 

98 

2 

0.891 

0.027 

0.010 

0.105 

0.907 

57 

89 

1 

0.881 

0.029 

0.01  1 

0.1  16 

0.892 

64 

78 

1 

0.869 

0.03 

0.013 

0.129 

0.88 

71 

72 

1 

0.857 

0.032 

0.014 

0.143 

0.869 

85 

68 

1 

0.845 

0.034 

0.015 

0.158 

0.848 

101 

52 

1 

0.828 

0.037 

0.019 

0.177 

0.824 

123 

35 

1 

0.805 

0.043 

0.029 

0.206 

0.793 

135 

31 

1 

0.779 

0.049 

0.032 

0.238 

0.777 

140 

30 

1 

0.753 

0.054 

0.033 

0.271 

0.77 

150 

27 

1 

0.725 

0.058 

0.037 

0.308 

0.757 

152 

23 

1 

0.693 

0.064 

0.043 

0.352 

0.754 

155 

21 

1 

0.66 

0.069 

0.048 

0.399 

0.751 
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to  give  a  linear  relationship  that  can  be  exploited  to  estimate 
the  parameters  {a,  h}  if  the  Weibull  distribution  applies  to 
these  data,  i.e.  if  the  log  of  time  t  versus  the  log  of  the 
cumulative  hazard  function  plots  as  a  straight  line,  then  a 
simple  graphical  means  results  to  visually  gauge  the  viabil¬ 
ity  of  the  Weibull  distribution.  Indeed,  linear  regression  can 
be  used  to  estimate  the  slope  and  intercept  coefficients  in 
Equation  (6)  via  the  technique  of  least-squares  analysis  to 
give  quantitative  estimates  of  the  straight-line  coeffi¬ 
cients.21  Figure  3  shows  the  ordered  pairs  (log(7/(t),  log(t)) 
computed  from  Table  3  entries  and  the  fitted  linear  regres¬ 
sion  model.  In  the  figure,  Hit)  is  estimated  by  summing  the 
non-parametric  estimates  of  the  hazard  rate,  h(t)  =  1  Ik, 
where  k  is  the  number  remaining  at  risk  at  time  t. 

Obviously  the  Weibull  distribution  describes  these  sur¬ 
gical  MTF  data  quite  nicely,  as  indicated  by  the  straight-line 
fit  and  near-perfect  correlation  value  shown  in  the  figure. 
Least-squares  estimates  can  be  transformed  as  indicated  in 
Equation  (6)  to  give  the  Weibull  parameter  estimates  indi¬ 
cated.  Next,  we  examine  the  maximum  likelihood  estimates 
(MLE)  of  {a,  b}  via  the  techniques  described  by  Nelson11' 
and  compare  the  parametric  model  based  on  the  Weibull 
distribution  with  the  non-parametric  Kaplan-Meier  esti¬ 
mates  given  in  Table  3.  An  approximate  confidence  interval 
will  also  be  presented  for  the  survival  function.  Before  pro¬ 
ceeding,  we  include  a  brief  overview  on  the  method  of 
maximum  likelihood. 

The  method  of  maximum  likelihood  seeks  to  determine 
the  unknown  parameters  of  a  distribution  to  maximize  the 
probability  of  obtaining  the  observed  set  of  outcomes.  A 
so-called  likelihood  function,  which  is  the  joint  probability 
distribution  of  the  known  data  and  the  unknown  parame¬ 
ters,  is  formed  and  parameters  are  chosen  to  maximize  this 
function.  The  approach  thus  provides  estimators  that  agree 
most  closely  with  the  observed  data.  The  likelihood  func¬ 
tion  is  typically  complex,  especially  for  applications  with 
censored  results,  and  statistical  software  is  often  required  to 
perform  the  associated  optimization:  we  used  the  SYSTAT 
package20  to  obtain  the  Weibull  parameters  in  Equation  (1) 
and  the  associated  survival  probability  entries  of  Table  3. 

Figure  4  shows  the  Kaplan-Meier  non-parametric  esti¬ 
mate  of  the  survival  function  and  also  the  estimate  based  on 
the  parametric  Weibull  model  (both  from  Table  3). 
Estimates  of  {a,  b}  via  the  linear  model  transformation  in 
Figure  3  agree  quite  well  with  the  MLE  values  of  Figure  4. 
Confidence  intervals  were  not  calculated  for  the  parame¬ 
ters,  but  we  suspect  the  interval  for  the  shape  parameter  b 
would  contain  the  value  1 .0,  indicating  that  the  exponential 
special  case  of  the  Weibull  woidd  adequately  describe  these 
results.  However,  to  be  consistent  with  the  smaller  shape 
parameter  values  for  MTFs  prior  to  surgery  as  observed  in 
the  MD  SME  panel,'  7  we  elect  to  standardize  on  the  two- 
parameter  version  of  the  Weibull  distribution  until  further 
results  are  available  for  analysis. 


Hazard  Plot 


log  (H(t)) 


Figure  3.  Log  hazard  plot  and  linear  regression  estimates  of 
Weibull  parameters  {a,  b} 


Figure  4.  Non-parametric  (Kaplan-Meier)  and  parametric 
(Weibull)  estimates  of  the  survival  function,  S(t) 

Given  the  almost  perfect  straight-line  fit  of  the  Weibull 
hazard  function  to  the  empirical  hazard  function  and  the 
nice  MLE  agreement,  both  graphical  and  analytical  meth¬ 
ods  suggest  that  the  Weibull  distribution  is  an  adequate  fit 
of  mortality  events  in  this  case.  For  completeness,  we  did 
conduct  a  chi-squared  goodness-of-fit  test  for  a  life  table 
representation  of  the  data  on  death  times  and  numbers  at 
risk  in  Table  3. 22  A  computed  value  of  3.63  was  obtained, 
and  when  compared  with  the  chi-squared  tabular  value  of 
5.99  with  two  degrees  of  freedom  and  an  a  of  0.05,  a  null 
hypothesis  for  the  Weibull  distribution  to  describe  these 
data  could  not  be  rejected. 

Figure  5  shows  the  approximate  95%  confidence  limits 
about  the  Weibull  S(t)  estimate.18  For  example,  the  proba¬ 
bility  of  survival  past  60  minutes  is  expected  to  lie  between 
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95%  Confidence  Limits  for  S(t)  -  Weibull  Model 


Figure  5.  Approximate  95%  confidence  limits  for  estimation  of 
the  Weibull  survival  function,  S(t) 

approximately  0.83  and  0.92,  where  the  mean  estimate  is 
about  0.88.  In  the  next  section,  the  fit  of  the  Weibull  distri¬ 
bution  to  the  CTR  data  is  graphically  compared  with  the 
results  from  the  MD  SME  panel. 

5.  Comparing  the  CombatTrauma 
Registry-based  Modeling  Estimates  with 
the  Expert  Panel  Results 

We  next  superimpose  curves  from  the  MD  SME  panel  on  to 
the  CTR-based  confidence  intervals  in  Figure  5.  The  two 
curves  labeled  ‘Panel-  ...’in  Figure  6  come  from  the  origi¬ 
nal  data  collection  effort  in  2003  for  a  high-risk  PC  and  the 
casualty  flow  case  ‘first  responder  to  STP/FRSS’.5  These 
curves  show  how  the  panel  expected  delay  times  of  30  and 
120  minutes  from  POI  to  surgical  MTF  entry  to  impact 
mortality  after  the  casualty  had  started  treatment  there. 


Figure  6.  Weibull  estimate  of  survival  function,  S(t),  with 
confidence  limits  and  expert  panel  short/long  delay  cases  for 
surgical  facility 


From  the  CTR  file  available  for  this  analysis,  times  from 
POI  to  STP/FRSS  entry  averaged  about  49  minutes  for  90 
records  that  had  complete  POI  and  surgical  facility  entry 
timing  data.  It  is  believed  that  the  large  majority  of  these 
records  corresponded  to  the  MD  SME  case,  with  no  inter¬ 
vening  treatment  after  the  first  responder  (i.e.  no  routing 
through  a  BAS  before  surgery  entry).  It  seems  logical  to 
expect  that  the  upper  MD  SME  curve  for  a  30-minute  delay 
would  shift  down  for  the  49-minute  empirical  delay  and  be 
closer  to  the  upper  CTR  confidence  limit;  we  make  no 
attempt  to  estimate  that  shift.  The  Weibull  curves  for  the 
49-minute  delay  look  very  comparable  in  basic  shape  and 
location  compared  with  the  MD  SME  bounds,  particularly 
for  the  entry  times  less  than  75  minutes. 

Some  caveats  are  presented  next: 

•  While  these  analyses  appear  to  confirm  the  Weibull 
model  and  the  survivability  estimates  are  in  a  rea¬ 
sonable  location  on  the  graph  shown  in  Figure  6,  we 
did  have  a  considerable  amount  of  missing  data  on 
exit  timings  that  were  estimated  by  NHRC  SMEs 
with  OIF  experience  in  the  medical  treatment  and 
evacuation  processes.  Some  38%  of  the  casualties 
were  missing  a  CTR  exit  time  that  was  added  manu¬ 
ally  from  field  reports  on  evacuation  events.  These 
missing  data  were  researched  and  resolved  before 
any  model  fitting  began. 

•  As  mentioned  at  the  outset,  the  quest  for  a  best¬ 
fitting  model  for  the  timing  of  combat  mortality  is 
ongoing  for  each  segment  in  the  casualty  flow  path 
of  treatment  and  evacuation.  Only  one  segment  was 
examined  here.  As  additional  injury  results  become 
available  in  the  NMC  CTR  EMED,  the  Weibull  dis¬ 
tribution  will  be  a  first  candidate  with  other  distri¬ 
butions  being  applied  as  warranted.  For  modeling 
and  simulation  purposes,  it  would  seem  desirable  to 
have  one  model  that  could  be  applied  to  all  segments 
if  an  adequate  single  representation  could  be  dem¬ 
onstrated.  It  would  not  be  necessary,  however. 

•  The  MD  SME  results  shown  here  are  averages  of 
the  responses  by  nine  MDs.  The  early  results  out  to 
about  75  minutes  track  very  nicely.  Starting  after 
75  minutes,  the  Weibull  curves  tend  towards  the 
higher  delay  curve  (120  minutes). 

•  No  attempt  is  made  to  perform  a  statistical  test 
of  goodness  between  the  MD  SME  panel  and  the 
empirical  results.  The  graphical  comparison  seems 
adequate  given  the  nature  of  the  two  sources. 

•  No  attempt  is  made  to  integrate  the  SME  results  with 
the  empirical  results  to  arrive  at  a  combined  best  esti¬ 
mate  of  the  Weibull  parameters  for  surgical  MTFs. 
Related  evolving  material  in  the  literature,  most  nota¬ 
bly  by  Singpurwalla23’24  and  Keller-McNulty  and 
Wilson,25  seems  applicable  but  is  beyond  the  scope  of 
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this  research  effort  at  the  moment.  It  was  our  objec¬ 
tive  to  demonstrate  how  comparable  the  two  results 
are  and  then  use  the  often  easier  to  acquire  SME 
opinions  for  areas  where  empirical  data  are  not  now 
available  such  as  the  early  interventions  in  Figure  1. 

6.  Summary  and  Future  Activities 

Our  main  objective  was  to  determine  whether  a  statistical 
model  could  adequately  describe  empirical  mortality  events 
in  a  casualty’s  medical  treatment  flow.  This  effort  is  directly 
related  to  confirming  the  efficacy  of  the  mortality  modeling 
approach  used  in  TML+.  Considerable  effort  was  devoted 
to  obtaining  complete  CTR  records  concerning  timing  of 
injuries,  either  at  the  POI,  or  entries  and  exits  at  MTFs. 
Although  not  reported  on  here,  we  also  spent  considerable 
effort  before  ISS  values  were  available  in  exploring  the  use 
of  data-mining  methods  to  identify  LT  injuries  from  the 
CTR  records;  the  interested  reader  is  invited  to  see  the  com¬ 
panion  NHRC  technical  report." 

Of  the  records  with  an  ISS  score  of  25  or  higher,  160  had 
both  entry  and  exit  timing  information  at  a  surgical  MTF 
and  were  judged  adequate  for  a  statistical  analysis.  As  only 
limited  path  data  were  available  on  when  the  injury  occurred 
and  where  a  patient  had  been,  the  effects  of  delay  time  in 
getting  to  a  surgical  facility  was  not  addressed.  Only  the 
one  path  through  a  first  responder  to  a  surgical  MTF  was 
deemed  adequate  to  consider. 

The  results  obtained  were  very  good:  the  probability  of 
survival  curve  (with  approximate  95%  confidence  limits)  in 
surgical  facilities  for  high-risk  patients  showed  a  strong 
graphical  agreement  with  results  inferred  from  the  2003 
MD  SME  panel.  Parameter  estimates  for  the  Weibull  distri¬ 
bution  were  made  using  both  graphical  and  analytical 
methods. 

While  the  results  apply  to  just  one  of  the  medical  inter¬ 
vention  types  in  TML+,  they  are  very  encouraging  for  two 
reasons: 

1.  they  strongly  reinforce  the  continued  use  of  the 
Weibull  distribution  for  use  in  TML+  in  surgical 
MTFs;  and 

2.  they  help  to  confirm  NHRC’s  reliance  on  special 
panels  to  provide  expert  opinions  when  empirical 
results  are  unavailable. 

As  the  comparison  of  empirical  and  MD  SME  results  is  so 
impressive  in  the  case  presented,  it  seems  justifiable  to  con¬ 
tinue  using  the  MD  SME  inferred  Weibull  modeling  results 
for  the  other  medical  intervention  facilities  and  mortality 
risk  categories  used  in  TML+  until  other  empirical  data 
become  available. 

The  CTR  data  set  continues  to  grow,  which  could  enable 
refinement  of  the  conclusions  in  this  paper  by  applying 


similar  analyses  to  other  medical  intervention  facilities  or 
operational  theaters.  A  few  possible  research  paths  include 
the  following: 

•  Considering  Figure  1  again,  it  would  be  very  inter¬ 
esting  to  have  early  mortality  data,  before  a  surgical 
MTF,  to  study  for  the  steeper  curves  that  more  dra¬ 
matically  impact  casualty  mortality. 

•  Any  delay  effect  from  the  POI  on  the  Weibull  results 
at  the  surgical  MTFs  was  not  examined  as  too  few 
cases  would  have  resulted  due  to  missing  entries  for 
the  time  of  injury.  It  seems  important  to  obtain  more 
empirical  data  to  look  at  this  important  concomitant 
variable  for  the  Weibull  model  in  surgical  facilities. 

•  These  results  were  obtained  from  OIF  CTR  wound- 
ings.  It  would  certainly  be  interesting  to  examine 
these  findings  with  Operation  Enduring  Freedom 
(OEF)  Afghanistan  injury  data  if  sufficient  records 
were  available. 

These  suggested  research  items  would  be  logical  next  steps 
in  pursuing  a  deeper  understanding  of  battlefield  mortality  sta¬ 
tistical  modeling  for  tactical  medical  logistics  studies  and 
analysis  and  future  decision  aid  developments.  We  also  intend 
to  pursue  other  more  general  objectives  as  described  below  to 
enhance  our  medical  network  modeling  and  simulation  tools 
and  input  data  sets  to  increase  their  validity  for  medical  plan¬ 
ning  efforts  such  as  force  sizing  and  resource  allocation. 

Medical  modeling  and  simulation  provides  a  continually 
growing  contribution  to  the  development  of  Department  of 
Defense  medical  policy,  campaign  testing  and  evaluation, 
technology  Research,  Development,  Test,  and  Evaluation 
(RDT&E)  and  medical  cost-benefit  risk  assessment.  As  the 
interest  in  medical  modeling  and  simulation  grows,  so  does 
the  complexity  of  the  tasks  to  which  it  is  applied.  Medical 
planners  now  routinely  use  TML+  and  related  tools  to  project 
medical  resource,  personnel,  and  transportation  requirements. 
However,  as  these  capabilities  become  accepted  as  the  norm, 
their  attention  is  increasingly  focused  on  more  complex 
issues  associated  with  global  cost  assessment,  crisis  action 
planning,  and  human  mortality  and  morbidity  projection. 
These  evolving  areas  of  interest,  however,  require  substantial 
investments  in  research  dollars  to  validly  define  and  quantify 
outcome  measures  in  complex  modeling  and  simulation 
tools.  The  NHRC  with  its  technology  partner,  Teledyne 
Brown  Engineering,  is  rapidly  moving  towards  the  develop¬ 
ment  of  the  empirical  data  necessary  to  support  accurate 
medical  outcome  projections  in  these  areas.  Recent  advances 
in  developing  knowledge  bases  sufficient  to  support  these 
evolving  initiatives  include  the  establishment  and  growth  of 
the  CTR  EMED  with  its  rich  casualty  medical  and  rehabilita¬ 
tive  outcome  data.  These  data  have  allowed  NHRC  to  project 
global  cost  assessment,  develop  crisis  action  planning  tools, 
and  project  human  mortality  and  morbidity  using  empirical 
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data  beginning  with  the  generation  of  the  casualty  at  the  point 
of  injury  and  continuing  on  to  rehabilitative  outcome. 

Disclaimer 

The  opinions  or  assertions  contained  herein  are  the  private 
views  of  the  authors,  and  are  not  to  be  construed  as  official, 
or  as  reflecting  views  of  the  Department  of  the  Navy  or  the 
Department  of  Defense. 
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